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An image sensor operable to vary the output spatial resolu- 
tion according to a received light level while maintaining a 
desired signal-to-noise ratio. Signals from neighboring pix- 
els in a pixel patch with an adjustable size are added to 
increase both the image brightness and signal-to-noise ratio. 
One embodiment comprises a sensor array for receiving 
input signals, a frame memory array for temporarily storing 
a full frame, and an array of self-calibration column inte- 
grators for uniform column-parallel signal summation. The 
column integrators are capable of substantially canceling 
fixed pattern noise. 
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INTEGRATED SENSOR WITH FRAME 
MEMORY AND PROGRAMMABLE 
RESOLUTION FOR LIGHT ADAPTIVE 
IMAGING 

This application is a continuation (and claims the benefit 
of priority under 35 USC 120) of U.S. application Ser. No. 
09/323,556, filed Jun. 1, 1999 now U.S. Pat. No. 6,057,539 
and U.S. application Ser. No. 08/867,835, filed Jun. 3, 1997 
now U.S. Pat. No. 5,909,026; and claims the benefit of 
priority under 35 USC 119(e)(1) of U.S. provisional appli- 
cation serial No. 60/030,646, filed Nov. 12, 1996. The 
disclosure of the prior application is considered part of (and 
is incorporated by reference in) the disclosure of this appli- 
cation. 

ORIGIN OF THE INVENTION 

The invention described herein was made in the perfor- 
mance of work under a NASA contract and is subject to the 
provisions of Public Law 96-517(35 U.S.C. 202) in which 
the Contractor has elected to retain title. 

FIELD OF THE INVENTION 

The present invention relates to semiconductor image 
sensors, and more specifically, to an imaging device with 
on-chip frame memory suitable for imaging applications 
under low lighting conditions. 

BACKGROUND OF THE INVENTION 

An active pixel sensor ( <- APS”) is a light sensing device 
with sensing circuitry inside each pixel. Each active pixel 
includes a light sensing element and one or more active 
transistors within the pixel itself. The active transistors 
amplify and buffer the signals generated by the light sensing 
elements in the pixels. In comparison with the widely used 
charge coupled devices (CCDs), an APS device has a 
number of unique and beneficial features. These features 
include the ability to receive and process input signals with 
the active pixels without the charge transfer process inherent 
in CCDs. An APS device is also compatible with CMOS 
processes. 

Elimination of the charge transfer allows APS devices to 
have a higher readout rate than those of CCDs and also to 
maintain their performance as the array size increases. 
Compatibility with CMOS processes allows many signal 
processing functions and operation controls to be integrated 
on an APS chip. Use of CMOS circuitry with APS devices 
also reduces cost of manufacturing and power consumption. 
Moreover, the active pixels of APS devices allow non- 
destructive readout and random access. 

One configuration of APS sensors is disclosed in U.S. Pat. 
No. 5,471,515 by Fossum et al., the disclosure of which is 
incorporated herein by reference. Such an image sensor 
typically operates at a constant finite frame rate with a 
constant signal integration time. The frame rate and the 
integration time are often preset for normal operating con- 
ditions under which illumination is usually sufficient. The 
performance of such a conventional image sensor suffers 
when the illumination is below the typical level to which the 
image sensor is configured. Since the frame integration time 
is usually preset for a predetermined normal light condition, 
little signal may be collected under a low light condition 
with an illumination level below the predetermined normal 
light condition. Conversely, too much signal may also be 
collected when input light level is higher than the predeter- 
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mined level. Thus, a clear image of a target can be difficult 
to obtain under different light conditions. 

One conventional technique to circumvent this problem 
under low light conditions uses averaging the signals from 
5 a plurality of neighboring pixels in order to reduce the noise 
level. This averaging technique improves the signal-to-noise 
ratio of the image sensor. The averaging process, however, 
also reduces the image resolution. Suppose a patch of NxN 
pixels are averaged, the signal-to-noise ratio is then 
10 improved by a factor of N 1/2 while the resolution is reduced 
by a factor of N. One limitation of this averaging technique 
is that the signal level achieved from averaging essentially 
remains unchanged compared to the signal levels of the 
selected neighboring pixels. Additional signal amplification 
15 is thus needed to obtain a brighter image. 

SUMMARY OF THE INVENTION 

The present disclosure describes a new approach to the 
above-addressed problem in image sensors. Instead of aver- 
aging signals from selected neighboring pixels, signals of 
the neighboring pixels are preferably added together accord- 
ing to the present invention. In contrast to the prior-art 
method, a sensing device in accordance with the invention 
increases both the signal and the noise with different factors 
so that the overall signal-to-noise ratio improves by the 
signal processing. The total signal can be increased by a 
factor of N 2 for a square patch of NxN neighboring pixels. 
The total noise comparatively increases by a factor of (N) 1/2 . 
, Q This allows the signal-to-noise ratio to increase by a factor 
of (N) 1/2 . One of the advantages of the present invention is 
significant enhancement in both the signal level and the 
signal-to-noise ratio. 

An APS image sensor with an adaptive resolution based 
35 on the above new technique is disclosed herein. One 
embodiment comprises an APS array for receiving input 
signals, a column buffer array for generating photo-induced 
electrical signals indicative of the input signals, a frame 
memory array for storing signals from the APS array, a 
40 column integrator array for signal summation for rows 
selected by a row decoder, and an output integrator for signal 
summation for columns selected by a column selector. 

One aspect of the APS image sensor of the present 
invention allows increasing the magnitude of the output 
45 image with a reduced spatial resolution under low illumi- 
nation. 

Another aspect of the APS image sensor of the present 
invention allows dynamically adjusting the image resolution 
and maintaining the output signal level above a preset level 
50 according to the conditions of a target object. 

Still another aspect is a self-calibration feature of the 
column integrator array that allows substantial elimination 
of the column-wise fixed pattern noise in the signals both 
prior to and during the signal integration. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other aspects and advantages of the present 
invention will become more apparent in light of the follow- 
60 ing detailed description, as illustrated in the accompanying 
drawings, in which: 

FIG. la is a schematic illustrating a preferred chip archi- 
tecture of an APS image sensor in accordance with the 
present invention. 

65 FIG. lb is a block diagram showing an exemplar 4-bit 
command word generator for light adaptive resolution 
adjustment, which is a part of a resolution control circuit. 
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FIG. 2 shows a signal chain of the APS image sensor of 
the present invention including the active pixel layout, the 
column buffer circuit, the frame memory circuit, the column 
integrator circuit, and the output integrator circuit. 

FIGS. 3a, 3b, 3c, and 3d show the equivalent circuit 
diagram of the column integrator during (a) calibration reset, 
(b) calibration sampling, (c)reset prior to integration, and (d) 
integration. 

FIGS. 4 a, 4b, 4c, 4 d, 4e, 4 f, 4 g, 4 h, 4 i, and 4/ show various 
electrical signals of an APS image sensor of the present 
invention in the time domain in a preferred operation mode 
for calibrating the fixed pattern noise. 

FIGS. 5a, 5b, 5c, 5 d, 5e, 5/ and 5 g depict various 
electrical signals of the preferred APS image sensor in the 
time domain in an example of programmable resolution 
readout process. 

FIG. 6 shows an image with full resolution taken with a 
tested image sensor in accordance with the present inven- 
tion. 

FIGS, la-lb and 8a-8b show signal enhancement by 
changing the kernel size in a tested image sensor in accor- 
dance with the present invention. 

DETAILED DESCRIPTION OF THE 
INVENTION 

The description of the present invention will use a CMOS 
APS device as an example for implementation. However, it 
should be understood that the method and design can also be 
implemented in other active pixel sensors, and may be 
usable in other image-acquiring systems. 

A chip construction 100 of the APS image sensor in 
accordance with the invention is shown in FIG. la. An active 
pixel array 110 with a plurality of active light-sensing pixels 
for receiving radiation from an target object is connected to 
a sensor row decoder 112 which operates to address rows of 
APS pixels. A column buffer array 120 has multiple buffer 
cells with each corresponding to a column of the APS array 
110. Each column buffer cell receives signals from an 
addressed active pixel of the APS array 110 to generate a 
signal. That signal is induced only based on the received 
radiation in that active pixel. 

A frame memory array 130 has memory cells with an 
one-to-one correspondence with the active pixels of the APS 
array 110. The radiation-induced signals of a full frame 
image captured by the APS array 110 are stored in an 
on-chip frame memory 130 prior to the signal integration. 
This on-chip frame memory 130 provides a convenient 
vehicle for light-adaptive imaging and can be further used in 
a time delay integration (“TDI”). A memory row decoder 
132 is used to randomly access and select rows of memory 
cells in the frame memory 130. 

A column integrator array 140 performs parallel signal 
summation for different rows of cells in the frame memory 
130. The summed signals are stored in a column memory 
array 150 which usually includes a series of capacitors 
corresponding to the rows of the frame memory 130. A 
column select circuit 160 is used to randomly access and 
select cells in the column memory 150 (equivalently, col- 
umns of the frame memory 130) to be added with one 
another by an output integrator 170. 

The APS chip 100 also includes a control circuit (not 
shown) for controlling the operation of the image sensor 
including operation of the clock generator circuits for 
synchronization, resolution adjustment, and the operation of 
the readout circuits. 
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The resolution of the output image can be determined by 
programmably changing the summation kernels in the frame 
memory 130 with the memory row decoder 132 and the 
column select 160. This operation can be controlled by a 
5 resolution control circuit (see FIG. lb). The resolution 
control circuit monitors the magnitude of the received signal 
level from the APS array 110 so that the size of the 
summation kernel can be determined for a subsequent signal 
processing. 

1° A threshold circuit with different levels of threshold 
values is used to determine the illumination condition and 
thereby adjust the kernel size. The smallest kernel size is one 
pixel for the highest resolution under good illumination. The 
resolution control circuit automatically increases the kernel 
15 size if the input signal decreases. 

FIG. lb shows a 4-bit command word generator 180 as 
one embodiment of the resolution control circuit. This 
circuit is used to adjust the resolution based on one frame of 
data by controlling the operation of the addressing circuit of 
20 the imaging array. 

The command word generator 180 includes a plurality of 
comparators 181-184 connected in parallel so that all 
receive the imager output. Each of the comparators has a 
different threshold level. FIG. lb shows four comparators 
181, 182, 183, and 184 with threshold levels V rh , 0.5 V r/> , 
0.25 V, ; „ and 0.125 V rh , respectively. Each comparator is 
connected to an associated digital counter and decoder 
185-188 to produce one of the bits of the 4-bit word. One 
way to determine a bit of a comparator is to assign “1” to a 
comparator if the signal is higher than the threshold value 
and “0” if the signal is lower than the threshold value. This 
allows the circuit to achieve 2 4 =16 different resolutions 
based on the 4-bit word. 

35 In one mode for the command word generator 180, one 
pixel data sampled at the output of the sensor array is 
compared with four binary scaled threshold voltages. Refer- 
ring back to FIG. la, the data may be sampled from the 
sensor array 110 or the frame memory array 130. The signal 
40 in each pixel of a captured frame is sampled and evaluated 
by the command word generator 180 in order determine the 
summation kernel size for a subsequent frame. 

One extreme case is that the signals from more than half 
the number of the pixels are higher than V th . Then the 4-bit 
45 word generated by the circuit 180 for each of these bright 
pixels is 1111. The control circuit may be programmed to 
indicate this case as a satisfactory illumination for the 
highest resolution depending on the requirements of an 
application. The resolution control circuit then configures 
50 the kernel size to be 1, i.e., no signal summation is per- 
formed and the resolution is the highest. Conversely, if the 
signals from more than half the number of the pixels are 
lower than the lowest threshold level (e.g., 0.125 V tA in FIG. 
lb), the 4-bit word is 0000 for each of these pixels. The 
55 control circuit may be programmed to indicate this case as 
a poor illumination. The resolution control circuit accord- 
ingly is controlled to set the kernel size to a preset maximum 
value and a predetermined lowest resolution will be used in 
the next frame readout. In adjusting the resolution, this 
go implementation has one frame latency. 

FIG. 2 shows a preferred signal chain 200 for the pre- 
ferred chip architecture of FIG. la. It should be noted that 
the capacitance values in the FIG. 2 only represent typical 
values of an embodiment and should not be construed as 
65 limitations of the invention. An active pixel 210 includes a 
photogate 211, a transfer gate 212, an output floating diffu- 
sion 213, a reset transistor 214, a gate source follower 215, 
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and a row selection transistor 216. Other design of the active 
pixel can also be used. An AC-coupled follower buffer 220 
represents a column buffer cell in the column buffer array 
120 shared by a column of active pixels in the APS array 
110. The input of the buffer cell 220 is clamped to a finite 5 
voltage VCLP by a clock CLP as the active pixel 210 resets 
and is released as the active pixel 210 gives out a signal. 
Thus, the buffer cell 220 converts consecutively-sampled 
APS sensor reset level and signal level into an optically 
induced signal that is exported to a memory cell 230 of the 
frame memory 130. Many aspects of the APS pixel 210 and 
the buffer cell 220 are disclosed in the above-incorporated 
reference, U.S. Pat. No. 5,471,515. 

The memory cell 230 includes a memory capacitor 231 
and a switch transistor 232. Circuit 240 is a switched- 
capacitor integrator and represents one cell in the column 15 
integrator array 140 with input from cells of a row in the 
frame memory 130 (e.g., memory capacitor 231). A folded- 
cascade operational amplifier can be used as amplifier 240a; 
though other amplifiers may also be used. A column signal 
memory capacitor 260 (CMC), representing one cell in the 20 
column memory 150, is AC coupled to the integrator output 
port as a load when signals are integrated. An output 
integrator 170 includes a compensated two-stage opamp 172 
with a large source-follower buffer to drive off-chip load at 
the video rate. 25 

A typical CMOS APS sensor uses column-wise source 
follower buffers, e.g., source follower 215 in the active pixel 
210 and the transistor 222 in the column buffer cell 220 
shown in FIG. 2, to sample signals onto the memory 
capacitors. A column-wise spatial fixed pattern noise (FPN) 50 
may be introduced by such sampling due to the threshold 
mismatch in the source follower transistors. The column 
integrator 240 in each column is designed to achieve an 
accurate signal summation by compensating for not only the 
opamp input offset but also the signal mismatch error 
prestored in the column capacitor cell 231. An appropriate 
switching sequence in combination with a feedback capaci- 
tor 251 (CC1), an extra capacitor 252 (CC2), and a clamp 
circuit 221 at the input of source follower 222 is used to 
autonomously accomplish this self-calibration process in the 
column integrator 240 at the beginning of each frame. 

The self-calibration process is preferably carried out in 
the following four steps: reset, sampling, reset prior to 
integration, and signal integration. In the reset phase, the 
mismatch signal AV is sampled on to the memory capacitor 4 _ 
cell 231 by switching on the transistor switch 224 (SW1). 
Transistor switches 242(SW3), 246 (SW7), and 247 (SW8) 
are turned at the same time, resulting in an effective circuit 
as shown in FIG. 3 a. For simplicity of discussion, it is 
assumed that the common voltage V c =0. At the end of the 5Q 
reset phase, the voltages at the input and output ports of the 
opamp 250 are V_=V + =V 0 =V tts , wherein V os is the offset 
voltage of the opamp 250. In the sampling phase, the 
calibration proceeds to switch off transistor switches 242 
(SW3), 247 (SW8), and 244 (SW5) and turning on the 
transistor switches 245 (SW6) and 241(SW2). FIG. 3 b 
shows the equivalent circuit of the integrator 240 in the 
sampling phase. Conservation of charges in the above two 
phases dictates that: 

c cu w + c ca v„=c CM v_ + c ccl (y_-v 0 ). (i) 60 

Assuming C cn =C cc1 =C and substituting V_=V + =V OJ , the 
calibration voltage at the output, V oc , can be expressed as 

V ^ V ^- hV . (2) 

This voltage is sampled onto the effective load capacitance 65 
302 (CL) at the sampling phase with a relative polarity 
shown in FIG. 3b. 


Next, a second reset is carried out as shown. By turning 
off transistor switches 245 (SW6) and 246 (SW7) and 
turning on transistor switch 247 (SW8), the capacitor 302 
(CL) is connected to the non-inverting input of the opamp 
250 with the flipped polarity. The voltages at the input ports 
of the opamp 250 become V + =V OJ - V 0 C =AV. The amplifier 
is reset again by switching on transistor switch 242 (SW3), 
resulting in an initial output voltage V 0 ,-=V_=V + =AV. FIG. 
3c shows the equivalent circuit. At the same time, an input 
voltage V ln 1 is sampled onto the memory capacitor cell 231 
along with the error voltage AV. 

The equivalent circuit for the final integration phase is 
shown in FIG. 3d. The charge transfer can be expressed in 
the following equation: 

C-(V I „' 1 +AV)=C-V_+C-(V_-V 0 _ 1 ), (3) 

wherein V 0 1 is the output voltage of the integration: 

V 0 ^2V_-(V inl +hV)^V-V inl . (4) 

The switching sequence for the self-calibration is illustrated 
in FIGS. 4a-4j. 

The above self-calibration is for a first input signal V in x . 
For a subsequent integration step with an input signal V I>J;2 , 
the corresponding output voltage after an identical switching 
sequence can be expressed as 

v 0i2 =2f_-(v;„,2+aV)-( v - + Lv,,i-aV)=ak-(v 1 .„ i1 +v;„, 2 ). (5) 

In general, the output voltage after nth step of integration is 

A (6) 

K,„ = AV V ilU . 

/=! 


Clearly, the offset V os of the opamp is eliminated from the 
integration process. The residual mismatch error AV is 
further eliminated by an extra clamp circuit 253 (FIG. 2) as 
the column integrator 240 resets prior to the integration by 
the output integrator 170. Therefore, a linear summation of 
the photon-induced signal from the active pixels in the APS 
array 110 (e.g., 210) is achieved with the preferred column 
integrator 240. 

Programmable resolution readout follows the self- 
calibration. With transistor switches 241(SW2) and 248 
(SW9) on, the column integrator 240 is reset by turning on 
242(SW3) momentarily. At the same time, the clamp circuit 
221 is turned on by switching on the transistor switch 
253(CLP1) to sample the clamped reset level of the inte- 
grator onto the column memory capacitor 260 at the output 
of the integrator 240. The switch 232 for selecting columns 
in each column is then turned on consecutively by the 
memory row decoder 132 for up to n rows if the kernel size 
is determined to be a square patch of nxn by the resolution 
control circuit. The signal from n rows are summed by the 
column integrator 140 and stored in the column memory 
array 150. Next, the transistor switch 248(SW9) is 
deactivated, and n consecutive columns are scanned after 
each reset of the output integrator 170 to add the signal from 
each kernel of n columns by an operation of the column 
select circuit 160. After all columns have been scanned, the 
next n rows are summed up in the same fashion. A full frame 
is read out after all the rows have been processed. Clock 
diagrams in FIGS. 5a-5g show the above-described 
sequence for a programmable resolution readout process. 

For an APS array with mxm elements, the resolution of 
the output image with a square kernel of nxn pixels is given 
by (m/n)x(m/n). Hence, the resolution is programmable by 
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varying the kernel size n based on the illumination condi- 
tion. The signal-to-noise ratio of the output image is 
enhanced by n times while the spatial resolution thereof is 
reduced by n times. In addition, the output signal level is 
increased by a factor of n 2 . 

An embodiment of the APS image sensor in accordance 
with the present invention has 32x32 pixels. The sensor can 
be fabricated by a CMOS process using 1.2 /.im n-well single 
poly and double metal with a linear capacitor option. The 
APS sensor pixel size is about 24 /<mx24 /<m. The frame 
memory cell pixel size may be approximately 24 /rmx28 f.im. 
The shape of each pixel cell in the frame memory may be 
slightly rectangular (i.e., the dimension along the column is 
longer) to achieve a memory capacitance of about 0.5 pf. 
The column integrators may be about 0.4 mm in a column 
pitch of about 24 /<m. A double metal process, “metal II”, is 
used in layout for column parallel routing because of the 
limited column width. Metal II is also used for light shield 
wherever possible in the frame memory array and column 
processing circuits. The total dye size of the chip may be 
approximately 2.8 mmx4.5 mm. 

TABLE 1 


Summary of Test Results 


Integrator linearity: 
Sensor saturation: 1 V 

better than 8n bits for 1 V swing 

Conversion gain: 

6 jiV/e 

Power consumptions 

1.5 mW @ 400 Frames/sec. 

FPN with calibration: 

15 mV 

FPN without calibration: 

40 mV 

Memory leakage: 

not observable in 1 second 


Eighteen out of twenty-four fabricated parts that were 
tested worked properly without any defective pixels. Table 
1 summarizes the test results. The linearity of both the 
column and the output integrators was better than about 8 bit 
in 1 V swing. The sensor had about 1 V signal saturation 
which was mainly limited by the dynamic range of the 
sensor pixel source-follower. The frame memory array had 
no observable leakage up to about 1 second. Total power 
consumption of the chip at a frame rate of 400 Hz was 
measured to be about 1.58 mW. 

FIG. 6 shows an image with full resolution taken at 100 
kPixels/sec with the tested sensor. With an error calibration 
activated, the column-wise fixed pattern noise (FPN) at full 
resolution was about 15 mV. This FPN did not increase as 
the signals from neighboring pixels were integrated. The 
residue FPN was found to be caused by the non-uniform 
charge injection. With the error calibration deactivated, the 
FPN was about 40 mV at full resolution. 

FIGS, la and lb demonstrate the signal enhancement 
when integration kernel changes from lxl pixels to 2x2 
pixels with the same frame integration time. Much better 
image quality is achieved with the FPN calibration (FIG. la) 
than without the FPN calibration(FIG. lb). At the lower 
resolution, a clear picture is achieved for an image hardly 
recognizable at the full resolution. FIGS. 8 a and 8 b show the 
detailed measurement, at a uniform illumination, of the 
signal (FIG. 8 a) and the signal-to-noise ratio enhancement 
(FIG. 8 b) as the summation kernel changes from lxl pixels 
to 4x4 pixels. A 60-mV signal from the single pixels is 
summed up to about 940 mV. The signal-to-noise ratio is 
enhanced following the square root rule as signals from 
different rows are summed. 

One unique aspect of the present invention is its light 
adaptive imaging. In particular, the preferred APS image 
sensor can autonomously and dynamically adjust the spatial 
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resolution of the output image therefrom to maintain a 
desired magnitude for an output signal in each output pixel 
according to the conditions of a targeted object. Specifically, 
as the illumination, i.e., incoming radiation from the object, 
5 becomes insufficient (i.e., below a threshold level) in main- 
taining a desired signal-to-noise ratio for a selected output 
image resolution with a given kernel size, the preferred APS 
image sensor will attempt to maintain the desired signal-to- 
noise ratio and enhance the output signal level by reducing 
10 the resolution of the output image with a properly increased 
kernel size. Conversely, if the illumination of the targeted 
object increases above the threshold level for the selected 
output resolution, the preferred APS image sensor will 
attempt to increase the output resolution while maintaining 
15 the reduced signal-to-noise ratio at the desired level by 
reducing the kernel size. 

Furthermore, this light adaptive imaging capability of the 
present invention allows for optimized imaging of a targeted 
object that moves with respect to the image sensor. One 
20 example of such an application is the imaging of a fixed 
target with an image sensor on aircraft or spacecraft. 

Since the aperture of the image sensor is usually fixed and 
the distance between the targeted object and the image 
sensor changes, both the field of view of the image sensor 
25 and the view angle of the object change with the target 
distance. If the aircraft or spacecraft is far away from the 
targeted object, the field of view and the view angle are 
small. This means the received amount of light emitted by 
the object is small and the needed resolution for truthfully 
.50 capturing the target image becomes low. The preferred APS 
image sensor can operate with a reduced output resolution 
by increasing the kernel size. This enhances the image 
brightness and the signal-to-noise ratio without sacrificing 
any image details of the targeted object. As the aircraft or 
35 spacecraft approaches the targeted object, the field of view 
increases and thereby the captured light from the object 
increases accordingly. Meanwhile, the view angle of the 
object increases and it is desirable to increase the output 
resolution of the image sensor for more detailed feature 
40 information of the object that is now available since the 
object is closer. Therefore, the preferred APS image sensor 
can reduce the kernel size to increase the output resolution 
while maintaining a sufficient brightness of the image with 
a signal-to-noise ratio above the threshold level. 

45 The inventors further recognized that the sensor of the 
present invention can be used for time delay integration 
(“TD1”). A sensor in accordance with the invention can 
effectively function as an TDI sensor by feeding the column 
integrated signal back to the memory array so that the signal 
50 can be summed with the data from a subsequent frame, with 
a time delay. This TDI signal processing may be used in a 
situation where the image sensor and a target object have a 
relative motion with respect to each other so that captured 
images in two sequential frames of the target object are 
55 shifted on the image sensor. For example, a feature of the 
object that is captured on the kth row in one frame may be 
captured on the (k+2)th row in a subsequent frame due to the 
relative motion. Accordingly, the signal in the kth row of the 
first frame will be added with the signal in the (k+2)th row 
60 of the second frame in summing the signals of the two 
frames in a TDI process. 

Although the present invention has been described in 
detail with reference to one embodiment, one ordinarily 
skilled in the art to which this invention pertains will 
65 appreciate that various modifications and enhancements 
may be made without departing from the scope and spirit of 
the present invention. 
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For example, any other image sensor, other than an APS 
array, may be used in practicing the present invention. In 
general, any voltage or charge mode readout sensor array 
may be used with the same scheme and circuits having the 
same functionality. Some examples of such image sensors 
include an CCD array sensor, a photodiode array sensor, a 
charge injection image sensor, and a hybrid focal plane array 
sensor. Also, the scheme disclosed herein can be used in all 
situations wherein signal integration is performed. 

Different type of amplifiers and integrators rather than 
those disclosed above can be adopted in practicing the 
present invention. 

Although the preferred sensor can be advantageous in 
light adaptive imaging applications, the sensor may also be 
used as a regular analog readout sensor when operating in a 
single resolution mode. 

In addition, any shape of pixel patches rather than a 
square shape may be used for a summation kernel, including 
a rectangular patch of pixels with m by n pixels (m*n). 

These and other variations are intended to be encom- 
passed by the following claims. 

What is claimed is: 

1. An adaptive programmable light imaging device, com- 
prising: 

an array of active pixel sensor pixels, each pixel produc- 
ing a signal based only on the received radiation within 
the pixel; 

a plurality of programmable summation kernels, each 
summation kernel programmable to selectively sum 
together a number of said pixels from said active pixel 
sensor; and 

a resolution control circuit, producing an output signal 
which controls a size of said summation kernels 
between a minimum value kernel size and a maximum 
value kernel size by monitoring a received signal level 
from individual pixels, statistically determining num- 
bers of pixels which are in specified states, and auto- 
matically changing the size of the summation kernels 
based on statistically determining. 

2. A device as in claim 1, wherein said resolution control 
circuit is based on illumination condition, and decreases the 
kernel size for better illumination condition and increases 
the kernel size for poorer illumination condition. 

3. A device as in claim 2, wherein said minimum kernel 
size is one pixel. 

4. A device as in claim 2, wherein said illumination 
condition is judged by a magnitude of said signal from said 
active pixel sensors. 

5. A device as in claim 2, wherein said resolution control 
circuit includes a digital circuit. 

6. A device as in claim 5, wherein said digital circuit 
includes a plurality of comparators, each having a different 
threshold, and wherein said signal from said active pixel 
sensors is input in parallel to each of said comparators. 

7. A device as in claim 6, wherein outputs of said 
comparators are connected to output respective signals. 

8. A device as in claim 2, wherein said active pixel sensor 
includes a photoreceptor and a buffer transistor and a selec- 
tion transistor. 

9. A device as in claim 8, further comprising self- 
calibrating the circuit prior to detecting an illumination 
condition. 

10. An adaptive programmable light imaging device, 
comprising: 

an array of active pixel sensor pixels, each pixel produc- 
ing a signal based only on the received radiation within 
the pixel; 
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a plurality of programmable summation kernels, each 
summation kernel programmable to selectively sum 
together a number of said pixels from said active pixel 
sensor; and 

a resolution control circuit, producing an output signal 
which controls a size of said summation kernels 
between a minimum value kernel size and a maximum 
value kernel size by monitoring a received signal level 
10 from individual pixels, statistically determining num- 
bers of pixels which are in specified states, and auto- 
matically changing the size of the summation kernels 
based on statistically determining further comprising a 

counter which counts a number of pixels which are in 
15 

specified illumination states and sets said summation 
kernel size based on said count. 

11. A device as in claim 10, wherein said counter detects 
whether at least half of the image has sufficiently bright 

20 pixels, and if so, configures the kernel size to be one. 

12. A device as in claim 10, wherein said counter deter- 
mines if at least half of the number of pixels are dimmer than 
a specified value, and if so sets the kernel size to a preset 
maximum value. 

25 13. An adaptive programmable light imaging device, 

comprising: 

an array of active pixel sensor pixels, each pixel having an 
in-pixel buffer transistor, and in-pixel selection 
transistor, and a photoreceptor producing a signal based 
only on the received radiation within the pixel; 

a double sampling circuit, operating to eliminate at least 
one amplifier offset from said signal; 

a plurality of programmable summation kernels, each 
35 summation kernel programmable to selectively sum 
together a number of said pixels from said active pixel 
sensor; and 

a resolution control circuit, including an illumination 
4Q condition detecting part connected to said active pixel 
sensor pixels and determining the illumination condi- 
tion therefrom, and producing an output signal which 
automatically controls a size of said summation kernels 
between a minimum value kernel size for a maximum 
45 illumination condition, and a maximum value kernel 
size based on a minimum illumination condition, 
wherein said illumination condition detecting part com- 
prises a counter which counts numbers of pixels which 
are in specified illumination states and sets said sum- 
50 mation kernel size based on said count. 

14. A device as in claim 13, wherein said counter detects 
whether at least half of the image has sufficiently bright 
pixels, and if so, configures the kernel size to be on. 

15. A device as in claim 13, wherein said counter deter- 
:o mines if at least half of the number of pixels are dimmer than 

a specified value, and if so sets the kernel size to a preset 
maximum value. 

16. A device as in claim 13, further comprising a frame 
memory, for storing an entire frame as a stored frame, and 

wherein said resolution control circuit is based on illu- 
mination condition in said stored frame, and decreases 
the kernel size in a subsequent frame for better illumi- 
nation condition and increases the kernel size in said 
65 subsequent frame for poorer illumination condition. 





